
. . i property ORGANIZATION 
T WORLD INTEU£CIVALP nalBurttU ^onPFRATION TREATY(PCn__ 

P tpaTION pubushed ^UNL^m^^ 



H04N 1/41, 7/26 



Al 



„„„__. Application Number: PCT/GB96/00623 
(M) I» te n»tlon»lApp 

(22) international FUingDaU: 



17 March 1995 (17.03.95) 



GB 



(30) Priority Data: 

9505469.8 

ifoi. IMPERIAL 

[GB/GB); Exhibition Road. Lonao 

mMnventors; and i >. BHARATH. Anil 

75 inventors/Applicants flbr g 'jff^ Medical Systems. 

ITT/GBV. Centre for C , B .' 0, °! Technology & Medicine, 
Krial College of Sconce ^ectaoWgy 
Sbition Road l^ndon SW7 ^ ^ ^ 



J *""" — - — ^ 

« ^C^W S C^C^DE?D^^^^^ , ^ , L^3yMDl 
2S§ ?V£ KP. KR. KZ. LK. Lfc l* ^ RUt SD , 
MOMK. MN. ^•J^i° T ? Z bA UG US, UZ. VN. 
SE SG. SI, SK^^^^-^sz UG). Eurasian patent 

Ga! GN*. ML, MR, NE> * 



Published 



Wir/.m»ern flI iona/s«rcyep^ lhe I 

amendment- 



TTr^eIsi^aT^^ 



(54) Title 



(57) Abstract 

A method for the progressive 
transmission of images from a server to 
^station comprises encoding *e 
imaee in multidimensional format, us 
ng m orthogonal wavelet transform. 
3 storing to encoded image 
ital file on the server. A user requo. 
n the image at a workstaaon sees, 
tot of all. a small thumbnail image, 
ff ir detail is required a request 
made to the server which sends * 
-deuil" image consisting onl, r of m- 
formation needed to upgrade the low 
Sutionimagetoahigher-resoluuon 

image. This approach saves transmit 
Tng across the network m ormauon that 
has 8 already been sent in connecuon 
with the original low-resolution image. 
U?systemUbeofpanicularu«m 

the hospital environment, for the trans- 
mission of medical images. 
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gROGREgglVj TRANSMISSION OF IMAGES 

-he - aversion relates tc progressive transmission cf 
^aaes, and in particular although net exclusively to tr.e 

•-^i^r. - - -nodical images across = 
™ a-essive transmission m— 

pouter network from a server to a workstation 

-, e ~ is often a need, within the ncsoit.x 
^vironmenc. for physicians and surgeons to have rapid 
:_ ss co a variety of medical images such as, o. 
Sample, x-ray images or CAT scan images. Typically 

.rnred h dicitai form on a centra, 
imaaes may oe storea ... 

^ver s"o that they may be called up as requireo -y 
us- on a local workstation or display client. 

-he oroblem with such an arrangement is tnat ,ne 



• v detailed medical 



transmission cf images, fc 

I^oes. across a n.c«or* puts a — ~ ; s 



In a realistic hospital 



ava : Table bandwidth. 

u taken for transmission of one 

environment, the time tatcen tor 

mailed image is likely to b- high oecause o .he 
competition for bandwidth arising from other users o, e 

svstem who may be attempting to 7-ew o - 

-he same time on other workstations. A 
^«*c»lty is that the user may wish to see onxy one 
- ae ^.s of oerhaps rather similar images, ana - c. 
l 0 S s l extremely inefficient to have to transmit eacn or 
' — ~ -co - -orsi^ before ;r.e user 

-hose imaaes, one by one, in fu-x -ecai- be.o. 
— - -, - w >-o^" res . 

<s abl- to choose the image ne or sne -c-u—/ ----- 
" one method of dealing with this problem is tc 
resent the user with a set cf small "thumbnail" images, 
sa r'-. cf wnicn -=» a -ow 

^ ~ .= r'^o S or- rre US6r _S 

- T.aaes Ci. c.*e ^ - - 

-*- 0 r-r S -he reauired image us:- — - 

-mages, the required image tnen oe::.: - 

"resolution form from the server to the workstation. 



BAD ORIGINAL 



WO 96/29818 



PCT/GB96/00623 



There are two ways in which the image information 
may be updated from a low -resolution thumbnail image to 
a high-resolution display cf the same image. The first 
way is tc retransmit the full content of the high- 
— solution imaae. This represents 5 wastage of 

bandwidth, since some of the image information is already 
present at the viewing console. The second way in wmch 
tine display may be updated is to transmit only the 
difference between the current image resolution and the 
deseed high-resolution image from the server to the 
workstation. In the case of a wide area distributed 
v'ewina svstem, the retransmission of full resolution 
'maaes'represencs a potential problem: for an individual 
image the retransmission of the full resolution image may 
not be significant; for the case in which there are many 
active viewing consoles, and a steady traffic of viewing 
and selection operations, the retransmission of the 
entire image may represent a significant bandwidth 
-ouirement over the transmission of the detail only. 

It is an object of the present invention at least to 
alleviate the problems of the prior art. 

is a further object to provide an efficient 
progressive transmission system having an improved 
"i smaller) bandwidth requirement. 

It is a further object to provide a system in which 
oroaressive transmission may be applied to any selected 
portion of an image that the user wishes tc see at 
higher- resolution . 

According to a first aspect of the present invention 
-He- * s orovided a method cf progressively transmitting 
Laces across a computer network from a server to a 
workstation, the method comprising: _ 
Kai storing an image on the server as a oigita^ 
r, le file comorising a low-resolution portion 
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reoresentative of a low- resolution version of the 
imaae, and a detail portion representative of detail 
in the image not contained within the low- resolution 
version ; 

(b) transmitting first information from the low- 
rescluticn portion to the workstation. and 
displaying the low-resolution version of the image 
at the workstation; 

; C ) selecting from the displayed image a user- 
defined area to be viewed at higher resolution; 
id) transmitting from the server second information 
from that part of the detail portion of the, file 
which relates to the user-defined area; 
(ei " combining at the workstation the second 
information and that part of the first information 
which relates to the user-defined area to create a 
higher-resolution version of the user-defined area; 

and 

(f) displaying at the workstation the higher- 
resolution version of the user-defined area. 
The method of the present invention is particularly 
a^ough not exclusively of use in the hospita. 
environment, for the transmission of medical images sucn 
as -chocardiographic images. There is often . 
requirements in a hospital environment to display a 
s-ie-On of mosaic of images, which may typically be 
<«oe slices of a 3D data set, or frames of an image 
s»au-nce. The present invention allows the individual 
'.maaes making up the mosaic to comprise smai_ iow- 
resoiution "thumbnail" images. A user who wishes to see 
-ore detail for cne particular image, simply ss.ec-.s tne 

- v„ o — rk-ra on t usina the mouse': . 

• mage ; for example cy c-CK-ng - 

r^-p. detail is then sent from the server, this further 
detail being combined with the information already heio 
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at the workstation to produce a slightly higher 
resolution image. Yet further information may be 
requested from the server if the user needs to see an 
even higher resolution version of the image. In each 
case, the only information that needs to be transmitted 
across the network is the detail information, that is the 
difference between the current low-resolution version of 
the image already held at the workstation, and the 
reauired higher-resolution version. 

In some embodiments, the information may be sent 
automatically from the server. In other embodiments, a 
specific request may be sent by the workstation to the 
server each time further information is required. This 
request may conveniently be user-generated. 

The preferred encoding method is to use the 
orthogonal wavelet transform. This is a transform which 
is discrete in both the time domain and in the scale - 
space (wavelet) domain. The small extent of the 
functions in the spatial domain allows for rapid 
convolution. The orthogonal nature of the functions 
used in the preferred embodiment guarantees that the 
minimum amount of information needs to be sent across the 
network. 

The user may select a desired area cf the image 
which he or she wishes to see in more detail. A desired 
region could for example be picked out on the screen with 
a mouse, preferably by dragging a box over the area of 
interest. If only an area cf the image is selected by 
-he user, the server sends information only on that 
selected area across the network. 

The invention extends to a computer network and/cr 
system for operating the method cf the present invention. 

The computer network may comprise a local area 
network or a wide area network, connected by wires or b 
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a wire-less link. The term "network" includes any means 
of remote communication between computers, and 
accordingly encompasses (without limitation} 
communication via the telephone system and/or satellites. 
5 Such means cf remote connection may be useful where for 

example the server is in a different hospital, and 
perhaps even in a different country, from the 
workstation . 

The invention may be carried into practice in a 
0 number cf ways and one specific embodiment will now be 

described, by way of example, with reference to the 
accompanying drawings, in which: 

Figure 1 is a block diagram showing, schematically, 
the operation of the discrete wavelet transform; 
5 Figure 2(a) shows the discrete wavelet transform 

decomposition process; 

Figure 2(b) shows the decimation process at 
successive scales ; 

Figure 3 shows the Daubechies 4, 12 and 20 scaling 
20 functions, evaluated at different respective scales; 

Figure 4 illustrates the decimation process in 

graphical form ; and 

Figure 5 shows how the mult i -resolution image may be 

stored. 

25 The preferred embodiment of the present invention is 

a method and/or system for the progressive transmission 
of medical images from a remote file server to a local 
workstation. Each image is stored on the server in a 
muiti-resoiuticn format comprising a low- resolution 
2Z reoresentat ion which may be called up as a sma-- 

"thumbnail" imaae on the workstation, and a plurality c: 
"detail" representations. If the user wishes tc see the 
image at lowest resolution, the thumbnail image alone is 
sent. If, having viewed the thumbnail image, the user 
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wishes to receive a higher- resolution representation, the 
first "detail" representation is passed along the network 
:o C he workstation. At the workstation, the "detail" 
representation is added tc the low- resolution image to 
orcvide the required higher- resolution image. If yet 
further resolution is required, the user can tall upon 
the next "detail" representation, stored on the server, 
which in a like manner is added to the information which 
has previously been passed to the workstation to provide 
an image at the next highest resolution. The process may 
be continued until the user has the image resolution he 
cr she requires, or has an image of the maximum 
resolution which is available from the server. 

The first stage in the process is, of course, to 
take an original image and to encode it in digital form 
in a digital data file which forms the muiti -resolution 
representation that is actually stored on the server. In 
order to split the original image up into a low- 
resolution image plus a plurality of "detail" 
20 representations or images, the preferred embodiment uses 

the orthogonal wavelet transform. 

The orthogonal wavelet transform is a discrete 
transform, similar to the Fouiier transform but instead 
of using sin functions wavelet mother functions are used 
instead. Each wavelet mother function has a limited 
extent in wavelet space, which contrasts with the sin 
functions used for Fouiier transforms which are of course 
infinite in extent. 

The wavelet transform has the following general 



form 



w = / :(:! T 
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The discrete wavelet transform, which is used here, 
is che same, except chat the values of a and b are 
• restricted to discrete values only. In the preferred 
: embodiment, orthogonal wavelets are chosen, for example 

the Daubechies wavelets shown in Figure 3. 

For any given signal, one can compute a series cf 
weights for discrete values of a and b which, when 
multiplied by the basis function and scaled leads back to 
the reoresentation of f. This can be done, in principle, 
without losses (other than those arising from rounding 
errors). The fact that the wavelets are orthogonal 
Guarantees the lowest possible error if certain terms are 
thrown away, for example if the user wishes to see an 
5 image of only intermediate resolution. 

The advantage of the present scheme is that the full 
set of coefficients or weights W(a.b) can be arranged 
into a multi-resolution representation of the original 
signal . 

As an illustration, consider the one -dimensional 
example shown in Figure 4. If one takes fin) to be a 
discrete signal to be approximated, one can obtain a 
first approximation by calculating f where f'.ni is 
calculated by determining the mean cf adjacent samples 
within f (n) . For example, f (0) is half the sum of f (0, 
and f(l). Likewise, f ' (I) is half the sum of f(2) anc 
f{3)> F « (n) may therefore be considered as an 
approximation g(n) to £(ni. One may also define a 
••detail" representation h(n) as the difference between 

f (ni and g (n) . 

The process may be repeated, decimating the sample 
each time, thereby creating a plurality of individual 
"detail" representations, each indicative cf tr.e 
difference between the current approximation and the last 
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previous approximation . 

To score the image in multi-resolution format, the 
final approximated signal is kept, along with ail cf the 
detail signals. 

The process may be represented schematically at each 
stage by the block diagram shown in Figure 1. At each 
stage, an approximation signal g(n) and a detail signal 
h(nj is determined from the input signal. The 
approximation signal is then fed back into the output and 
a further approximation and a further detail signal 
obtained. At an appropriate stopping point, the final 
verv low- resolution g(n) output is retained, along with 
ail cf the detail outputs h(n). 

The structure shown in Figure 1 can be used for 
mult i -resolution orthogonal biorthogonal , or quadrature 
mirror filters (QMF) where each of the above cases is 
specified by the chosen filter coefficients. 

The filters are simply denoted as low pass, g(n) and 
high pass, h(n), to present a general form, but in fact 
these filters are related to scaling function, <f>(xi, and 
wavelet function ^(x) [2]. The decomposition process is 
illustrated in Figure 2. 

Each block has both a low pass filter and a high 
pass filter. The output of each filter is decimated by 
a factor of two. The resolution of the output cf the low 
pass filter is also changed because cf losing high 
frequency detail . 

The output of the filter chain, A-'*S, signal 
represents a discrete approximation of signal at the 
resolution 2" r ' . The ether output is called the detail 
signal, D-'^'S , at the resolution 2". If the eric inci- 
sional has N samples, then the tctal number cf samples ir. 
the detail and approximation signals is also N . Thus the 
discrete wavelet representation has the same total number 
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cf samples as the original signal- The smoothness of A2 : S 
depends on the shape cf the scaling function. In general 
we would wish to have a smooth scaling function, but this 
function should satisfy some renditions such as 
orthogonality, support width etc. Dne cf the important 
things is the spatial support width) cf the scaling 
function which divides the wavelet transform into compact 
into compact and non-compact wavelets. 

In compactly supported wavelets the scaling function 
and the wavelet function have mere regularity at the 
expense cf a large number of non-zero coefficients of 
hin. , which results in a large support for <Mxj . This 
support width increases linearly with their regularity in 
some wavelets such as the so-called Daubechies wavelets 
{Daubechies, 1988) . As shown in Figure 2, increasing the 
number cf coefficients of h(n) , leads to an increase in 
the smoothness of the scaling function. Note that the 
three traces of Figure 3 have been evaluated at 
resolution in inverse proportion to their support, so 
that the x axis does not allow comparison cf true 
support. Because of the compact support and 

orthogonality the Daubechies wavelets provide a suitable 
basis set. The Maliat basis set is constructed from 
splines in continuous space, then discretized, and hence 

is not compact. 

A multi-resolution decomposition is a method cf 
representing signals at different scales cf 
magnification. The concept of muiti -resolution 

decomposition was initially developed by Maliat [2] to 
address the problem of characterising image scenes in a 
scale independent manner. The approach cf Ma^j.at is 
essentially a pyramidal decomposition, along the lines or 
Burt and Adelson, but uses values of h;n; and g;n) sucn 
that the decomposition is onto an erthegonai basis. The 
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basic idea is that of successive approximation, together 
with adding detail signal from one approximation to the 
next. Assume that we have a ladder of spaces such that 

... c r. z V. c y_ c V,. c V_ r ... 



With the property that if 

f (X) € V. 

then 

f . ;■; -2 : k) 6 V. ; ke I ; and f<2x) 6 V. 
If W. is the orthogonal complement of V. in V... so we 

have 

V... = V. © W. 



Thus, W. contains the detail signal necessary to go 
from V. to V.... If we have an approximation of a signal 
at a resolution corresponding to V : , then a better 
approximation to the signal is obtained by adding the 
detail signal corresponding W : . This detail signal is the 
projection of the signal in W : . 

By interpolating the output 



of the decomposition depleted in Figure ' 2 ) , one obtains 
an approximation 
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S In) 



the signal Sir.) at a resolution 2' : . The interpciat 
defined by the following equation: 



S •:.•:)* £ <t> (r.-k) SCO 



■1) 



wnere 



a;., s 



denotes insertion of a zero between each sample of th 
discrete signal 



This interpolation is a convolution between r. 1 
scaling function o and 

a:.- s 



This particular interpolation represents an examp 
in going from a representation of the signal at one sea 
zo an approximation of the signal at the sea 
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immediately above it. If we wish to go from a 
representation of the signal at, say, a scale of 2" : to ar. 
apDroximat ion of the signal at some scale 2" : * : , where 
1>I , iz is necessary to dilate the scaling function 
5 appropriately. 

The appropriate dilation may be performed as 
follows: if the scaling function of Equation il) is at 
the resolution 1(1. e. 2" : ) , then it belongs to V. . 

0 Since 



r . c v*. 



3 



one may write 

k 

(2) 

This equation may be generalised to relate the 
scaling function at a scale 2 : to the scaling function at 
a resolution 2 : ' : for any j. Therefore, 



s (-) = 53 * (2; " n ' k) A i sik) 

2 : K 



For the sake of simplicity, the above examples have 
assumed a one -dimensional signal rather than a two- 
dimensional image, which of course is what we are really 
interested in. In dealing with images rather than 
sianals, it is necessary to use the corresponding two- 
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dimensional discrete wavelet transform !DWT! . The two- 
dimensional transform may again be represented by a 
repeated function block where, instead zi a pair of 
outputs, we have four outputs corresponding :s the low- 
frequency image component and tne .-.igr.-rreouer.cy 
vertical, horizontal and diagonal components. The latter 
three imaae components correspond to the detail signais. 

For details of the two-dimensional decomposition, 
refer to Mailat [1] . It is possible to generalise 
Eouaticn (3) to the two-dimensional case: 



5 r..r.) = ££<t>(2--- 1 il-ic)4)(2-- 



•4 j 

due to the separability of the two-dimensional 
decomposition . 

15 Figure 5 shows, in schematic form, how the 

information may be stored in a data file on the server. 
The lowest resolution image that is expected to be 
required (A) is stored, along with the sequences of 
detail information D. , D, and D, . If, fcr example, the 

2 0 lowest resolution image A is an 8 x 3 image, one will 

automatically obtain from the algorithm three sets of 
detail signals each cf which also has size 3x3. The 
next level up is 15 x 16, and so on. To produce an image 
of 15 x 15 resolution, the three detail signals Z : are 
25 added into the low- resolution image A. Similarly, to 

create an image of 32 x 32 resolution, the detail images 
Z- are added onto the previously-created 15 x 15 image. 

It will cf course be understood that it is not 
necessary for the data file to be stored exactly as it is 

3 1 shown in Figure 5. All that is required is that the low- 

resolution image can be selected from the file and sent 
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across the network to the workstation followed, if 
requested by the user, by each of the detail images in 
order . 

In looking at medical images , it is often useful to 
ce able to view a particular region cf interest in 
creater detail. The infrastructure of orthogonal, 

rr.ult iresoiut ion image decomposition provides an efficient 
way cf doing this, in terms cf bandwidth requirements. 
Consider a region cf interest, such as a heart valve on 
a cardiac image. By selecting an approximate region cf 
interest on a low resolution, interpolated image, the 
cresent infrastructure allows one to view that region at 
full resolution without needing to increase the 
resolution outside of the specified region. 

The user first selects an area cf interest (for 
example using a mouse! , and that area is then redrawn on 
the screen in greater detail. For example, if the user 
wishes to view in more detail only the top left-hand 
corner of the thumbnail image A, he or she simply selects 
that area and a signal is sent to the server instructing 
it to send further detail appropriate to that area only. 
With the file structure shown in Figure E, the server 
would send only the detail information contained within 
the top left-hand corners of each of the three D. images. 
If the user then wishes to view that enlarged image in 
vet mere detail, the server would send only the top left- 
hand corners of the D : images . 

In order to refine the detail over a particular 
user-selected area cf the image, we need to determine the 
range of influence of all samples (pixels » within the 
area we are interested in, over all scales cf the 
decomposition process. This range cf influence is 
clearly a function cf the width of the filters used, that 
is the impulse response cf the filters used in the 
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decomposicion . Since we are typically using filters of 
very small length, this range of influence is fairly 
localised throughout all space. However, proper 

reconstruction can be carried out only if we can 
5 correctly identify all of the samples, at each scale, 

which influence the desired image within the selected 
area . 

A general implementation involves tracing all of the 
influenced children of a pixel throughout the various 

0 components of the image in the transform space. 

Although tedious, this is quite within the capabilities 
of the skilled man. If orthogonal filters are used, as 
they are in the preferred embodiment, then the range of 
influence simply becomes the direct children (4) of each 

5 pixel. 

This allows great savings in bandwidth. Consider 
the requirement to view a region of microcalcif ication in 
a digital x-ray mammogram at a resolution of 256x256 
pixels. Mammograms typically have full pixel dimensions 

0 of 4096x4096 pixels. In such a situation, immediate 

enhancement of the region requires ail the coefficients 
to go in resolution from, say a 512x512 icon to a full 
resolution region is less than 1% of the total 
coefficients necessary to rebuild the entire image at 

5 4096x4096 resolution. 

The results of our study show that the 
reconstruction of an image from its full wavelet 
decomposition represents by selecting an appropriate set 
of scales represents a valid method of building a muiti- 

C resolution framework for progressive transmission cf 

echocardicgraphic images. We have shown that not only 
does this framework support the progressive updating cf 
"thumbnail" images, but also that smooth interpolation cf 
these small images by an appropriately dilated scaling 
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function lead to low resolution images at magnifications 
(scales) comparable to the original image. Such blurred 
images may be used, for example, to select with precision 
a particular region of interest, which cne wishes to view 

5 in greater detail -'i.e. at full resolution;. 

The clear advantage of adopting this approach to the 
clinical viewing cf images in a networked client /server 
environment, such as a teleconferencing cr remote 
diagnosis application, is that communications bandwidth 

C is used optimally. Where a large number of clients are 

involved, and commercial networks are employed, the 
savings can be significant. Finally, even on local area 
networks using ethernet technology, the cut in overheads 
can represent a significant saving in bandwidth. 

5 
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CLAIMS : 

1. A method of progressively transmitting an image 
across a computer network from a server to a workstation, 
the method comprising: 

is; storinq an image on the server as a digital 
file, the file comprising a iow-resoiut ion portion 
reoresentative of a low-resolution version of the 
image, and a detail portion representative of detail 
in the image not contained within the low- resolution 
version ; 

!b) transmitting first information from the low- 
resolution portion to the workstation, and 
displaying the low-resolution version of the image 
at the workstation; 

(c) selecting from the displayed image a user- 
defined area to be viewed at higher resolution; 

(d) transmitting from the server second information 
from that part of the detail portion of the file 
which relates to the user-defined area; 

(e) combining at the workstation the second 
information and that part of the first information 
which relates to the user-defined area to create a 
higher-resolution version cf the user-defined area; 
and 

(f) displaying at the workstation the higher- 
resolution version of the user-defined area. 

2. A method as claimed in Claim 1 in which the detail 
cortion of the file comprises a plurality of detail sub- 
cortices, successive sub-portions containing information 
or. successively finer detail within the image. 

3 . A method as claimed in Claim 2 
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information from a first sub-portion is transmitted from 
~he server, the third information being combined at the 
workstation with the first information to create a first 
intermediate resolution version cf the image. 

4. A method as claimed in Claim 3 in which fourth 
information from a second sub-portion is transmitted from 
the server, the fourth information being combined at the 
workstation with the first intermediate resolution 
version of the image to produce a second intermediate 
resolution version, of higher resolution than the first 
intermediate resolution version. 

5. A method as claimed in Claim 4 in which fifth 
information from a third sub-portion is transmitted from 
the server, the fifth information being combined at the 
workstation with the second intermediate resolution 
version of the image to produce a third intermediate 
resolution version, of higher resolution than the second 
intermediate resolution version. 

6. A method as claimed in any one of the preceding 
claims in which the digital file is encoded using a 
transform having a basis function cf finite extent:. 

7 . A method as claimed in any one cf Claims 1 to 5 in 
which the digital file is encoded using the wavelet 
transform. 



8 . A method as claimed in any one cf Claims I tc 5 ir. 
which the file is encoded using the crthogcnal wavelet 
transform. 



9. A method as claimed in Claim 5 in which the 
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crthogonal wavelet transform uses a Daubechies' wavelet. 

i: . A method as claimed in any one cf the preceding 
clairr.s in which the combining at the workstation includes 
interpolation . 

11. A method as claimed in any one of the preceding 
claims including storing a plurality of images on the 
server, and displaying on the workstation a corresponding 
plurality of low-resolution versions of the images for a 
user to select. 

12. A method as claimed in any one of the preceding 
claims in which information from the detail portion is 
transmitted from the server on receipt of a request from 
the workstation. 

13. A method as claimed in Claim 12 in which the request 
from the workstation is user-selectable. 
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